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Abstract 



Research into several aspects of robot-enabled reconnaissance of random fields is reported. The 
work has two major components: the underlying theory of information acquisition in the exploration of 
unknown fields and the results of experiments on how humans use sensor-equipped robots to perform 

T ^ a simulated reconnaissance exercise. 

The theoretical framework reported herein extends work on robotic exploration that has been 

\Q reported by ourselves and others. Several new figures of merit for evaluating exploration strategies are 

proposed and compared. Using concepts from differential topology and information theory, we develop 
the theoretical foundation of search strategies aimed at rapid discovery of topological features (locations 
of critical points and critical level sets) of a priori unknown differentiate random fields. The theory 
enables study of efficient reconnaissance strategies in which the tradeoff between speed and accuracy 
can be understood. The proposed approach to rapid discovery of topological features has led in a natural 



way to to the creation of parsimonious reconnaissance routines that do not rely on any prior knowledge 
of the environment. The design of topology-guided search protocols uses a mathematical framework 
that quantifies the relationship between what is discovered and what remains to be discovered. The 
quantification rests on an information theory inspired model whose properties allow us to treat search 
as a problem in optimal information acquisition. A central theme in this approach is that "conservative" 
and "aggressive" search strategies can be precisely defined, and search decisions regarding "exploration" 
vs. "exploitation" choices are informed by the rate at which the information metric is changing. 

The paper goes on to describe a computer game that has been designed to simulate reconnaissance of 
unknown fields. Players carry out reconnaissance missions by choosing sequences of motion primitives 
from two families of control laws that enable mobile robots to either ascend/descend in gradient 
directions of the field or to map contours of constant field value. The strategies that emerge from 
the choices of motion sequences are classified in terms of the speed with which information is acquired, 
the fidelity with which the acquired information represents the entire field, and the extent to which 
all critical level sets have been approximated. The game thus records each player's performance in 
acquiring information about both the topology and geometry of the unknown fields that have been 
^h randomly generated. 

Index Terms 
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I. Introduction 

The use of mobile point sensors to explore unknown fields is of current interest across many 
characteristic length scales and in many domains of science and technology. Applications ranging 
from environmental monitoring [QQ|, where mobile sensors must traverse significant distances to 
map variations in thermal fields or concentrations of chemical species, to nano-scale imaging [2], 
where the main tool is the Scanning Probe Microscope, which acquires an image by scanning 
a point probe over micrometer- scale samples. Previous work has been focused on utilizing field 
characteristics (length and time scales) to design optimal reconnaissance strategies [OD], 0, (4), 
0. In these references, the reconnaissance agents are distributed and controlled such that they 
minimize the error in their estimate of the unknown field. 

The goal of the reconnaissance strategies described in the present paper is to map topological 
features of unknown random fields. This feature-based reconnaissance accommodates unknown 
field reconstruction aimed at capturing the field's geometry (level sets, curvatures, gradients, 
etc.) and the field's topology (critical points). It exploits hierarchically organized feedback loops, 
wherein low level reactive motion primitives as described in [6], [7], [8], O, ifTOll . [fTTTl allow the 
mobile sensor to track features, while at a high level, models of optimal data acquisition guide 
the system to efficiently assemble a global representation of important qualitative features of the 
field. To this end, we make a connection between differential topology and information using 
the language of ergodic theory and the notions of entropy that were first applied to ergodic 
theory by Kolmogorov. Following essentially the same development that is presented in lfT2l 
and r T3ll for ergodic mappings, we shall define the entropy of a scalar field by means of a three 
stage process. We begin by defining the partition entropy of finite partitions of compact sets. 
Next, we define the entropy of a function on a compact domain with respect to a finite partition 
of its range. Ultimately, this leads to a definition of the entropy of the function itself. One of 
the main elements in our information-theoretic approach to the study of unknown fields is the 
relationship between the entropy of the field and a certain finite partition of the domain called 
the topology-induced partition. This sought-after connection between the differential topology 
of the field and its entropy is given in Theorem 3. 

The relationship between topology and entropy guides our exploration of reconnaissance 
strategies. Recalling previously published work [6], Q, J8]|, two families of sensor motion 
control laws are introduced in terms of which reconnaissance strategies can be implemented. 
These families consist of (a) motions that climb or descend gradients of the unknown field and 
(b) motions that follow lines along which the value of the field is constant. By appropriately 
switching between the control laws, data is acquired about the unknown field. As more and more 
lines of constant value (isolines) become known, they provide the boundaries between cells in an 
increasingly fine set of partitions of the domain that we call data-induced partitions. By keeping 
track of the rate of increase of data-induced partition entropy as isolines are mapped, we obtain 
essential insight into how effectively the reconnaissance strategy is discovering information about 
the unknown field. 

While every isoline that is mapped increases the entropy of the data-induced partition, it is 
only certain isolines that yield information about the topology-induced partition. Specifically, it 
is shown that by mapping a new isoline in a cell in the data-induced partition with negative Euler 
characteristic, new information regarding the topology-induced partition may be obtained. Using 
this observation, a reconnaissance strategy is described in which our robotic motion primitives 
instantiate the well known exploration versus exploitation paradigm. Specifically, the gradient 
ascend/descend motions are used to discover local maxima and minima in the field. The search 
for these constitutes the exploration phase of the reconnaissance. The localization of max's 
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and min's enables the exploitation phase in which isolines can be mapped so as to provide an 
increasingly complete set of information regarding the topological characteristics of the field. 
One can think of the isoline mapping strategy that is designed using knowledge of the fields' 
extremum points as providing a steepest ascent along an information gradient aimed at learning 
as much as possible about the topology induced partition. 

A decision criterion based on the rate at which an information metric is increasing under isoline 
mapping informs choices of whether to next map isolines or gradients in the proposed reconnais- 
sance strategy. The criterion involves parameters similar to those in simulated annealing, and a 
comparison is made between aggressive strategies (which place high value on finding extremum 
points) and conservative strategies (emphasizing isoline mapping to fill in geometric detail 
regarding the field). Using Monte-Carlo simulations, reconnaissance strategies are compared 
in terms of how rapidly they acquire information about the topological characteristics of the 
unknown field. 

In the final part of the paper, we turn to the study of human performance in the kinds of 
reconnaissance missions we have abstractly characterized. Work reported in lfT4l suggests that 
in exploring unknown environments humans assimilate information in the form of topological 
maps. The same circle of ideas will play a role in our determination of styles emerging in 
the human guided reconnaissance of unknown fields. A computer game that simulates a human 
guided robot-enabled search is described. The game is designed so that a human mission director 
can have a robot map either gradient ascending/descending lines or map isolines of an unknown 
field. A metric of bias toward the acquisition of topological information is proposed, and in terms 
of this metric, we assess the styles of twenty-seven subjects who played the game. All subjects 
exhibited some bias toward acquiring topological information, but the range of the bias was 
broad. The players who sought and discovered information about the topology-induced partition 
tended to be very parsimonious in terms of the numbers of isolines that were mapped. They also 
tended toward consistency in their performance from one game to the other. 

The road map of this paper together with the progression of the main concepts is provided in 
Fig. [Tj In the next section, we present a mathematical abstraction of scalar fields based on their 



critical point structure. In Section [III} we show how reconnaissance protocols for uncovering 
the field's topological structure can be based on the motion primitives developed in [6], Q, 
(U. This is further formalized in Section [IVJ where the reconnaissance process is described as 
the acquisition of information as measured by a certain Shannon-like entropy metric. In Section 
[VJ we propose protocols for unknown field reconnaissance and compare them by Monte-Carlo 



simulations. Human in-the-loop reconnaissance is treated in Section VI, where we describe an 



experiment in which human subjects play a computer game of simulated robotic reconnaissance. 



In Section VII, we offer some concluding remarks and discuss open problems. 

The following notation will be used throughout the paper. Notation that is contained within a 
single section is omitted for simplicity. 

• Information 

- H(Vk) — The entropy of V k 

- H(M\Vk) — The conditional entropy of M with respect of 14 . 

- H(V k )— The bound of H(M\V k ) given H(V k ). 

• Reconnaissance 

- /(r) — the scalar field for reconnaissance. 

- X — The reconnaissance domain. 

- b zso (r ) — An isoline mapping motion program that starts from r 

- b 9rad (r ) — An gradient line mapping motion program that starts from r 
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Sec. IVA 



Sec. II 

Topology of Potential field 

• The Topology Induced Partition 

(TIP) 

•The Poincare-Hopf Theorem 

applied to TIP: Thm. 1&2 



Information theory of 
potential fields 

• Partition entropy and its 
properties: Prop. 1-3 

• The entropy of potential 
fields: Thm. 3 



Sec. IVB 



Reconnaissance as information 
acquisition 

• Reconnaissance as iterative 
information acquisition 
•The Data Induced Partition 
(DIP) and its properties: Prop. 4 




Sec. Ill 

Motion primitives based 
Reconnaissance 

• Motion primitives syntax 

• General Reconnaissance 
protocol 



^ 



Guiding the reconnaissance through 
information 

• Inferring the TIP through the DIP: Thm. 4 

• Efficient reconnaissance based on the 
topology of the cells in the DIP: Cor. 4 

• Reconnaissance through motion 
primitives as exploration vs. exploitation: 
Cor. 5 and Thm. 5 



Sec. IVC 




Protocols for robotic 
reconnaissance 

• Provably convergent protocol: 
Thm. 6 

• Monte Carlo comparison with 
naive scanning strategy 



Analysis of human guided 
reconnaissance 

• Computer game for supervised 
robotic reconnaissance 

• The human preference 
towards discovering the 
topology of potential fields 



Sec.V 



Sec. VI 



Fig. 1. A schematic of this paper organization and concepts progression. 



- B k — A sequence of k motion programs. 

- S(B k ) — The set of all mapped level sets, including extremum points, by B k . 

- Q(B k ) — The set of all mapped gradient lines by B k . 
Topology 

- £ — A level set of /. 

- ( — A gradient line of /. 

- Cr(/, X) — The critical level sets of / within X. 

- Cr 0,2 (/, X) — The critical level sets of / within X that are single points (maxima and 
minima). 

- Cr^/, X) — The critical level sets of / within X that are contours (corresponding to 
index 1 critical points). 

- .M (/, X) — The topology induced partition of / within X. 

- V k — A data induced partition after the execution of k motion programs. 

- V£— A cell in V k . 

- x(Yk) — The Euler characteristic of the set V k . 
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- V' k — A subset of Vk containing cells with Euler characteristics smaller or equal to — 1. 



II. Topology of a scalar field 

The following section presents a formal treatment of the topology of scalar functions defined 
on compact planar domains. In particular, we present a construction called the topology induced 
partition of a function / : R 2 —> R, which is based on the concept of monotonic sets that can 
be traced to the image processing literature l(T5lh lfT6lh ifTTll . Here, however, the description of 
the topology is defined in the context of the data acquisition particulars of the search process. 
In lfT8ll . we have introduced the related notion of a monotone search sequence. These concepts 
and their relationship to reconnaissance decisions will be discussed in what follows. 

The function under consideration is a scalar field, / : X —> R, defined on a compact, connected 
and simply-connected domain IcK 2 called the search domain. To avoid pathological behavior 
which would not contribute to the current discussion, the following technical assumptions are 
imposed on /: 

Assumption 1: The function / : X —> R is a Morse function. 

This assumption implies that its critical points are non-degenerate, and therefore isolated. 

Assumption 2: The boundary of the search domain, dX 9 is a level contour of /, f\$x =const. 

Basic results in Morse theory lfT9l allow us to describe the topological characteristics of 
functions that satisfy these assumptions. Conditions under which a random field will be a Morse 
function are given in [20] . To make the analysis invariant to scaling, we assume that range of 
the unknown function always reflects the full dynamic range of the sensor being modeled. 



A. The topology induced partition 

To investigate the topology of scalar fields through Morse theory, we will consider these 
functions as surfaces in three dimensions. Then, as in [19], / : X — >> R can be considered as the 
height function, and important topological characteristics can be described through the changes 
in the number of connected components of 



X c = {r G X : /(r) > c] 



(1) 



as c is allowed to vary over the range of /. 

Assume without loss of generality that the range is the unit interval [0, 1]. Then as c decreases 
from 1 to 0, we observe in Fig. [2] that the number of connected components of X c changes at 
the critical points of the height function — connected components appearing at the maxima (Fig. 



2(a)| and |2(b)| ), merging at the saddle points (Fig. |2(c)| ) and disappearing at local minima (not 
shown). 






(a) 



(b) 



(c) 



Fig. 2. The height map for three different values 
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Our objective is to study the differential topology of the surface by decomposing it into 
component parts, each of which belongs to a homotopy class that is a member of a family of 
homotopy classes determined by the critical point structure of /. We obtain a corresponding 
decomposition of the domain X into diffeomorphic components. Diffeomorphisms preserve 
homotopy classes and the Euler characteristic invariant. Recall that for a connected set, the 
Euler characteristic is defined as x =the number of vertices —the number edges+the number 
of faces for any triangulation of the set II2TT1 . For a disk, the Euler characteristic is 1, and it is 
easy to see that introducing a hole reduces the Euler characteristic by 1, since this effectively 
removes a face from the simplicial decomposition. Hence for an annulus, the Euler characteristic 
is 0, and other planar domains and their Euler characteristics are illustrated in Fig. [3j 



(a) X = 1 (b) X = (c) X = "I 

Fig. 3. For any planar domain with sufficient regular boundary, the Euler characteristic is 1 — ^ where g is the number of 
holes. 

What we show next is that there exists a unique decomposition of the surface into critical 
level sets and annuli. 

A well-known result from Morse theory is the following. 

Lemma 1 ([19]): Define for some arbitrary c\ < c 2 the sets X C1 and X C2 associated with the 
height map according to ([T]). Assume that there is no critical value c* s.t. c* G [ci, c 2 ], where a 
critical value is defined as a value for which there exist a critical point r*, /(r*) = c*. Then, 
X C1 is diffeomorphic to X C2 . 

From this lemma, it follows that we may partition the surface into connected components 
having the same Euler characteristic between the critical values of the function. 

Definition 1: For any domain V C X, we denote the set of connected components of V by 

cc(y). 

Definition 2: A critical level set of the function / in the domain X is a connected component 
£(c*), /(r*) = c*, with r* being a critical point, which satisfies 

£(c*) G cc({rGl:/(r) = c*}) 

r* G f(c*). 

The set of all critical level sets of / in the domain X will be denoted by Cr(/, X). In addition, 
let Cr°' 2 (/, X) denote critical level sets of dimension (extremum points — i.e. values of / at 
local maxima and local minima), and let Cr^/, X) denote critical level sets of dimension 1 
(contours associated with saddle points), so Cr(/,X) = Cr 0,2 (/, X) UCr 1 (/,.X). 

Finally, define a set of annular subsets of the domain X, 

M(f,X)=cc(X\Cr(f,X)). 

We call M the topology induced partition of the function /. We note that M is strictly speaking 
not a partition, since the critical level sets are removed, and hence the cells are open sets. Fig. 
[4] shows the topology induced partition of a particular field. The white areas are magnifications 
of critical level sets. 
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Fig. 4. The topology induced partition of a particular field. 



B. Properties of the topology induced partition 

To establish the properties of the topology induced partition, we use the Poincare-Hopf theorem 
to relate the number of critical points to the Euler characteristic of the set in which this function 
is contained (see Def. 1 for the notation in the theorem). As noted, the surface can be viewed as 
a manifold embedded in three dimensional space. Thus, the gradient of the field, V/, induces a 
corresponding vector field on the surface. In view of these observations, we restate the Poincare- 
Hopf theorem in the following manner. (See [22]|.) 

Theorem 1 (Poincare-Hopf): Let / : X —> R be as above satisfying Assumptions [T] and [2] in 
a compact domain X C M 2 . Let V C X be a connected set with a boundary dV having the 
property that each connected component of dV is a level set of /. Then, if m is the number of 
the extremum points (index and index 2 critical points) of / in V, and n is the number of its 
saddle points (index 1 critical points), 



m — n = x(Y)' 



(2) 



The next corollary shows how the Poincare-Hopf theorem can be restated in terms of the 
critical level sets, Cr(/, X). 

Corollary 1: Let the set V C X be connected with the connected components of its boundary 
being level sets of a Morse function, / : X —> R, and let m and n be respectively the cardinalities 
m = \{£ e Cr°' 2 (/,X)|C C V}\ and n = |{f e Cr^f.X)^ c V}\. Suppose that with every 
critical set, £ e Cr x (/, X), there is only a single associated saddle point. Then m — n = x(V r ). 
Proof: Let the curves £o?£i? • ' ' ?6 be the connected components of 9]/, having the same 
orientation such that in traversing £ in the positive sense, int(V) lies inside £ > while in traversing 
£j, j = 1, •••,/, in the same positive sense, int(l/) lies outside of £j. That is to say that V is a 
compact multiply connected domain whose outer boundary is £ and whose interior boundaries 
are £i, • • • , &. Then, x(V r ) will be equal to 1 — /, since every £ J9 j = 1, • • • , / will correspond 
to a hole in V\ 

Denote by m^, rii the numbers of even index and odd index critical points respectively in 
the interior of each boundary curve, int(^). Each int(^) is diffeomorphic to a disk, and we can 
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apply Theorem [TJ Moreover, taking into account that £ is the outer boundary, we can write: 



i 

m = m — 2_, m i 
i=i 
i 

n = no-y^rtj 

2=1 



and therefore 



m — n = tuq — Uq — V^(m^ — m) = 1 — I = xiV)- 



2=1 




Fig. 5. A particular topology induced partition and its graph representation. Critical level sets correspond to vertices and cells 
correspond to edges. 



To further elaborate the properties of the topology induced partition, we recall that each 
cell Mi e M is an annular component, and therefore has two boundaries one of which may 
degenerate to a point. Thus, a cell can be viewed as connecting two level sets. As a result, 
we will be able to associate the topology induced partition to a tree graph, G(v,E), where the 
vertices correspond either to critical level sets or to the boundary of the search domain, dX. The 
edges of the graph will be associated with the annular cells of the topology induced partition. In 
this way, the leaf vertices will correspond to extrema or to the boundary, dX. The vertices with 
degree 3 will be associated with the elements of Cr x (/, X). Such graphs are referred to as Reeb 
graphs, [15 j. Fig. [5] illustrates the graph representation of a simple topology induced partition. 

The Reeb graph representation implies the following cardinality for the topology induced 
partition. 

Theorem 2: Let / : X -> [a, b] satisfy Assumptions [I] and [2j let Cr°' 2 (/, X) C Cr(/, X) be the 
set of all extremum points of / within X, and suppose that for each element of Cr x (/, X) there 
is a single associated index 1 critical point. Then, for the topology induced partition, M(f,X), 
the following holds: 

|M(/,X)| = 2|Cr°' 2 (/,X)|-l. 

Proof: Since G(E, v) is a tree graph, 



\E\ 



-l = |Cr(/,X)| + l-l, 
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where it is taken into account that the boundary is also a vertex. However, Corollary [T] gives the 
following relationship: 

\a?*{f,x)\-\a x {f,x)\ = i. 

This yields 

\E\ = \M(f,X)\=2\0°*(f,X)\-l. 

■ 
The graph representation will be further utilized to illustrate different aspects of the proposed 
reconnaissance strategies. 

III. Reconnaissance with motion primitives 

A motion primitive for unknown field reconnaissance denotes a feedback control law that 
allows the vehicle to navigate the field to track either a path along a level set, or a path 
of increasing or decreasing intensity. Thus, the vehicle can map features of the surface and 
accumulate information about its general topology, and about its topology induced partition in 
particular. 

In the design of the reconnaissance strategy, we consider two motion programs, b lS0 (r ) and 
b 9rad (r ), that map respectively isolines, £, and gradient paths, £, passing through the point r . 
One can ultimately think of a motion program as a construction that utilizes feedback control 
laws to track a specific feature of the unknown field. In [6J, Q, (8), we have shown how isoline 
and gradient tracking control laws can be designed that rely purely on the intensity measurements 
of the field. 

The b lS0 (r ) will produce the level set contour that passes through the point r . The b 9rad {r ) 9 
on the other hand, will produce a contour passing thorough r that is tangent to the gradient at 
each point, and whose end points either lie on the boundary, or at one of the extremum points 
of the function. 

A reconnaissance protocol will be a rule for choosing points r and the motion programs 
that should be executed from them. Thus, a program b lS0 (r ) will consist of two steps: i) go 
to the point r , and ii) map the isoline passing through r . A sequence of k motion programs 
will be denoted by B k = {b u 6 2 , • • • , b k \, where b { e {b iso , b grad } (ie{l,2,..., k}) and every 
program has its own originating point r l . In what follows, the reconnaissance protocols that will 
be discussed will be restricted to choosing the originating points for every consecutive motion 
program from the paths traced by a previously executed program. In more detail, the different 
features mapped as a result of a string B k will be denoted by 

S(B k ) = {&,&,-..,£„}, (3) 

for the collection of mapped level sets and extremum points, and by 

Q(£? fc ) = {Ci,C 2 ,---,0}, 

for the mapped gradient paths. Then, in terms of this notation, the proposed reconnaissance 
protocol is depicted in Fig. [6} The originating points for the isoline following motion programs 
will be chosen from the set of mapped gradient paths, and vice versa, the originating points for 
the gradient following motion programs from the set of mapped level contours. 

We illustrate this reconnaissance protocol by a particular example. Consider a scalar field with 
a topology induced partition such as the one illustrated in Fig. |7(a)[ The critical level sets are 



depicted as dashed curves, but these are unknown at the initiation of the search. Initially the 
only level set that is known is the boundary of the domain. We assume that the agent choses two 
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Choose Q e Q(Bk 
Choose r e Q 
Execute b lS0 (r ,X) 



Choose £i G S(Bk 
Choose r £ ^ 
Execute b grad {v Q ,X 




Fig. 6. The general protocol of applying the b tso and the b grad motion programs into search strategies. First, a particular 
mapping program is chosen (isoline or gradient line mapping). Then, if the program is isoline mapping, r G is chosen from a 
previously mapped gradient line, and vice versa, if the program is gradient line mapping, r is chosen from a previously mapped 
isoline. The mapped object is then added to the map. 




(a) 



(b) 



(c) 



Fig. 7. Example application of the b %so and the b grad motion programs according to the proposed strategy. 



sufficiently separated random points on this bou ndary and maps the gradient lines that originate 
from them with the b 9rad motion program (Fig. 7(b) ). 

From a topological point of view, these actions can be described in terms of the graph 
description of the topology (Fig. |8(a)| ). That is, the tw o ma pped gradient lines correspond to 
paths in this graph that span a connected subgraph, Fig. 8(b) , In effect the b 9rad primitives have 
revealed the existence of two maxima and the saddled point that is implied by them (Thm. [Tf. 
Now assume that the robot proceeds and maps the isolines also depicted as red curves in Fig. 



7(c)[ with originating points chosen, according to the proposed protocol, at random locations 
along the previously mapped gradient lines. As a result, the robot has confined one of the saddle 
points within the shaded region of the figure, i.e. it has collected information about the topology 
induced partition of the function. Here, the term information is used loosely, but in the next 
section it is formalized, in order to provide a framework within which the reconnaissance can 
be equated to information acquisition. Moreover, subsequently in this work, we formalize and 
generalize the type of Poincare-Hopf based inference that allowed us in the example to conclude 
the existence of a critical set associated with a saddle point. 



IV. Reconnaissance and information 

In this section, we introduce the information theory of scalar fields through first defining 
the concepts of entropy and conditional entropy of partitions. The main result of this theory 
is summarized by Theorem [3} which establishes that the scalar field information content is 
determined by its topology, which, on the other hand, is represented by the topology induced 
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(a) 



(b) 



Fig. 8. The inference of a part of the topology from the mapped gradient paths. The two valence three nodes in (a) represent 
unknown index 1 critical level sets. In (b), the circled nodes correspond to mapped local maxima, and the boxed valence three 
node corresponds to the inferred index 1 critical point (depicted within the shaded region in Fig. 7(c). 



partition. In Section |TV-B[ we show that pursuing the discovery of this partition through the data 
acquisition protocols specified in Section [HI] corresponds to iterative information acquisition. 
Section |TV-C| builds upon the information formulation of reconnaissance and the properties of the 
topology induced partition described in Section [II] to state rules through which the reconnaissance 
can be effectively guided based on topology inference procedures and information metrics. 



A. Measuring the information capacity of a function or field 

Information about an unknown field will be measured by the entropy (analogous to Shannon 
entropy) associated with certain partitions that reflect how important qualitative features of the 
field vary over the search domain. Work along these lines that studied transformations on measure 
spaces in terms of metrics on partitions of their domains was reported in lfT2l . Our work is 
inspired in part by this early effort, as well as by the concepts of information in ergodic theory 
as reported in [13] and the numerous references cited therein and the seminal work on topological 
entropy by Adler, Konheim, and Mc Andrew, E3ll . 

We begin by discussing the notion of partition entropy, roughly following the development in 
lfT3ll . Suppose that X is a compact metric space — e.g a compact, connected subset of W 1 having 
nonempty interior. Suppose that X comes equipped with a measure /i together with a a-algebra 
E of measurable subsets. Let a and f3 be two at-most-countable partitions of X such that all cells 
in a, (3 are members of E. To make the link between partitions and probability and information 
theory, we think of the partitions a = {A\, . . . } and )3 = {£?i . . . } as describing the possible 
outcomes of experiments, with /jl(Aj) being the probability of outcome Aj in experiment a and 
fi(Bk) being the probability of outcome B k in experiment /?. 

To each partition, we associate a measure H(a) (resp. H(/3)) that describes the amount of 
uncertainty about the outcome of the experiment. We thus define the partition entropies 

H ( a ) = ~ ZUea M^) lo g2 MA), and 

(We assume that /jl(X) = 1.) In the same spirit, the conditional entropy of a conditioned on )3 
is given by 
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Bje/3 AiGa rv " ,j/ f'\ JJ J> 



_ E , (B))E M^) log: 



BjE/3 Ai£a 






-EE^n^iog,^^!. ( 4) 



To establish certain properties of the conditional entropy, we state the following definition. 

Definition 3: Let )3 and 7 be two partitions of X. We say that 7 is a refinement of )3 (denoted 
by writing f3 < 7) if each element of f3 is the finite union of elements of 7. The partition 7 is 
said to be a proper refinement of /3 if there is an element B e /3 that is the union of no fewer 
than two elements of 7. 

Then, the following bounds on the conditional partition entropy (cf. [24 j) are included for 
completeness. 

Proposition 1: Let a and )3 be two partitions of a given domain X, then the metric H(a\j3) 
defined by ([4]) satisfies the inequality 

0<H(a\/3) <H(a), (5) 

with equality: 

H(a\P)=Q 

if and only if /3 is a refinement of a (/3 > a). 

Proof: Starting with the right side of ([5]), the log- sum inequality G4ll yields: 

= -n(Ai)log 2 n(Ai), 

and therefore 

#(«!/?) < - X) MA)log 2 M^i) = //(a). 

For the left side, on the other hand, 

with equality satisfied if and only if Bj C A (or more precisely /jl(Bj C\Ai) = 0), and therefore, 

afAinBi) 
log 2 W ' ' ; < 0, 

which when substituted back in ([4]) concludes the proof. ■ 

Proposition 2: Let a = {Al, . . . }, /3 = {5i, . . . }, 7 = {Ci, . . . } be partitions of X such 
that /3 is a refinement of a. Than H(a\j) < H(f3\j). The inequality is strict if (3 is a proper 
refinement of a, provided that 7 is not a refinement of /3. 
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Proof: Let A <E abt written as 

n A 

A={jB j . 
We claim that 

_ , (A n o) log2 a_i < _ g „ w n o lo g2 «^ 

with the inequality being strict if ua > 2. The result is easily established for ua = 2 by writing 
Pi = n(B 1 n C)//jl(A n C), p 2 = A*(-B 2 n C)/p(A n C) and g = p 1 + p 2 - Then we have 

n ^ Pi i Pi P2 1 P2 
< log 2 log 2 — 

9 9 9 Q 

with the inequality being strict unless one of pi and p 2 is zero. This inequality is easily shown 
to be equivalent to (|6]) for ua = 2, where the inequality is strict unless /i(Bj D C) = 0. 

The proof of the case ua > 2 uses a simple inductive argument along the same lines. Here, 
for each C G X we have 

-^MAnC)i og2 ^^<-^M^nC)io g2 ^^ 

with the inequality being strict if /? is a proper refinement of a. The proposition follows by 

summing both sides over C G 7. ■ 

Corollary 2: Let a = {Al, . . . } and /3 = {Si, . . . } be partitions of X such that /3 is a 

refinement of a. Then i7(a) < H(f3). The inequality is strict if /5 is a proper refinement of a. 

Proof: This is a special case of Proposition [2] in which 7 is the trivial partition 7 = {X}. 

■ 
Proposition 3: Let a = {Al, ...},/? = {£?i, . . . }, 7 = {Ci, . . . } be partitions of X such that 
7 is a refinement of /3. Then ii(a|/3) > if(a|7). 

Proof: For each £?j G /3, write Sj = (J^ Q and note that 



^ B j) = J2^( C i)i aild 



2=1 

no 



//(A fc n 5j) = 5^/x(A fc n a),VA fe e «. 



«=i 



The log-sum inequality |24l| may be rendered 

E^ncoio g2 ^^ > (j>(A fc n CO j^ ^gf^ 

Multiplying both sides of the inequality by —1 and summing over the cells in the partition yields 

the desired result. ■ 

Remark 1: Proposition [5] shows that conditional entropy, H (a |/3), is a non-increasing function 

of /3 under partition refinement. Even when 7 is a proper refinement of )3 9 however, it is not 
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necessarily the case that H(a\j$) > H(a\j). A simple example that illustrates this is given by 
taking a = /3 = {X} and letting 7 = {Ci, C2} where /x(Ci) = /x(C 2 ). 

We seek to transfer these notions of entropy to quantify the entropy of continuous scalar 
functions defined on compact domains. Let X C R m be a compact, connected, simply connected 
domain, and let / : R m —> R. Then f(X) is a compact connected subset of E which we write 
as [a, b]. At the outset, we fix a finite partition, II m , of this interval: 

a = Xq < x\ < • • • < x m = b. (7) 

For each Xj, j — 1, . . . , to, we denote the set of connected components of / _1 ([xj_i, Xj]) by 

CCC/" 1 ^^-!,^^)). 

For any such partition, we obtain a corresponding domain partition 

m 

Vn rn = \J{cc(f- 1 ([x J _ 1 ,x J ]))} (8) 

of X. We define the entropy of / with respect to Vn m = jV 1 , • ■ ■ , V N } (or equivalently with 
respect to II m ) as 

where /i is Lebesgue measure on R m . We shall also refer to ([9]) as the partition entropy of / 
with respect to Vn m (with respect to II m ). 

It is easy to write down expressions for the entropy of simple functions as the one illustrated 
in Fig. [9J In all cases, the range is the interval [0, 1], and we partition this range into m equal 
subintervals of length 1/m. We evaluate the entropy ^ for m between 1 and 20. The entropy 
grows in all cases as the number m of cells in the range partition increases (See the bottom 
graph of Fig. [9]). However, it grows with different rates, with function (a) growing with log 2 m, 
(b) growing slower since it contracts lengths near and expands them near 1, and (c) growing 
with the highest rate since its topology partitions the domain into finer sets. We shall thus be 
less interested in the specific value of the entropy than in the relative sizes for different classes 
of functions — and in how these scale as m becomes large. At this point, we can quantify the 
relationship between the topology induced partition and the entropy of the function. 

Theorem 3: Let V m be a domain partition of X corresponding to a range division of / : X —> 
R into m equal components. Let M be the topology induced partition of the same function and 
define Si for each cell Mi e M as: 

s = suPreM, /(r) - inf rGM , /(r) 
su PrGx/( r ) -inf rGX /(r) ' 
Then the following holds: 

lim (#(/, V m ) - log 2 m) < H{M) + V ^T ^g 2 S h (10) 

ra^oo ^— ^ fl(X) 

where n = \M\ is the cardinality of the (finite) topology induced partition. 
Proof: For each M { e M, let 

V = {V e V m : V j C M,}, 
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5 10 15 

Range partition elements 



Fig. 9. From left to right, the functions mapping the unit interval onto itself are given by (a) fi(x) = x, (b) J2{x) — x 2 , 
and (c) f^{x) — sin 2 2ttx. The bottom graph corresponds to the entropy, H (/, Vn m ), for the three functions for uniform range 
partitions of different sizes, m 



and let 



Then, we may write 



V* = {V j eV m ' V j is not confined in any Mi}. 



H(fv )- V V ^ J) w ^ yij) V ^w ^ yk) 



ti*) 



H(X) v ^ /i(X) w M (X) ' 



Given any e > 0, we can find an m e sufficiently large that for all m > m, 



E E 



M^') , _ rtV*) 



, KX) 



log. 



^(X) 



< 



E 

MiGM 



fJL(Mi] 



KMi) 



fi(x) log VWM + 2 



and 



- V ^io S f^l<i 

^ KX) l0g2 AX) < 2' 



y/e G y* 



(11) 



(12) 



(13) 



where [•] denotes the ceiling function. The right hand side of ( |T2] ) is: 

^ ) + g^ log2M + £ = ^ ) + g^ log2 IM + log2m + l. 
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Then combining these relationships back in ( pT) yields 

H{f, V m ) - log 2 m < H{M) + E 4^ lo & — + e 

As m —> oo, ^^ — » Si, and since e can be chosen arbitrarily, the result follows. ■ 

Remark 2: Under the assumption that X is compact and / : X —> R is Morse function, the 



right-hand- side of (10) is well defined and the limit on the left-hand- side is always finite. 



Remark 3: There exist fields for which either one or both sides of the inequality ( fTO) ) are 
negative. 

Remark 4: It is not difficult to prove that if II m is any partition @ of the range of the function 
/ having corresponding domain partition Vn m ®, then 

lim (H (/, V n J - log 2 m) = lim (if(/, V m ) - log 2 m) , 

provided 

max |xj — Xj_i| — >> 0. 

l<j<m 

The last remark motivates the following. 

Definition 4: Given a compact set X and a Morse function / : X —> R, and given a set of 
partitions of the range of / into m equal subintervals, the entropy of / is defined by: 

lim (H(f,V m ) -log 2 m). 

Remark 5: For any partition II m ([7]) of the range of / having corresponding domain partition 
Vn m ([8]), we can think of the conditional entropy of the domain partition conditioned on the 
range partition being given by the expression 

If II m is the uniform partition with Xj — Xj-i = ^ for j = 1, 2, . . . , m, then, up to an additive 
constant that is independent of m, this conditional entropy coincides with 

H(f,V m ) -log 2 m. 



Strictly speaking of course, ( fl4| ) is not an actual conditional entropy since Vn m and II m are 
partitions of different spaces that do not have a common measure. Moreover, as indicated by 
Remark [3j the expression ( [T4| ) may take negative values, which means that it cannot be viewed 



as a conditional entropy in the usual sense. Nevertheless, it is useful to think of the quantity ( fT4] ) 
as the limiting value under partition refinement of the conditional entropies of domain partitions 
corresponding to increasingly fine sequences of range partitions. (Another way to view ([14]) is 
as the capacity of the function / to act as an information channel between the range, [a, 6], and 
the domain, X.) 

Going back to Sec. II, the bound of the entropy of a particular function, as its range partition 
is refined, depends completely on its critical level sets and their scalar value. Therefore, the 
reconnaissance strategies, that we will analyze, will be focused on the discovery of the topology 
induced partition. 
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B. Reconnaissance as information acquisition 

We note that the reconnaissance cannot directly discover the boundaries of the cells in the 
topology induced partition that are associated with saddle points. For this to happen, the robot 
should choose an originating point, r , for an isoline mapping primitive that lies on a particular 
critical level set £* e Cr x (/, X), Cr x (/, X) being the set of critical level sets associated with 
saddle points. Since these are zero-measure sets, their discovery in the course of random mapping 
of isolines can occur with zero probability. Therefore, we pursue a procedure by which the set of 
mapped level sets, S{B k ), can be used to make inferences about the topology induced partition 
and more specifically about Cr 1 (f, X). To achieve this, by analogy with the domain partition 
associated with a given range partition, we describe the collected data as a partition induced by 
the mapped level sets (contours and extrema), V (S (B k )) := V k given by 



V k :=cc(X\S(B k )). 



(15) 



The elements of this data induced partition will be connected components that do not contain 
mapped level sets, and we will denote them by V k = {V k , V k , • • • , V k }. 

The evolution of V k under the search process corresponds to iterative partition refinement. 
That is, given the mapping of a level set £ in the set Vj?_ ± G Vk-u the updated data induced 
partition will be given by: 

V k = (V*_i \ {V^}) U cc (VU \ i) , (16) 

where again cc(-) signifies the set of connected components. (Fig. [To| shows an example update 
of the data induced partition under the mapping of a particular isoline.) We also remind that 
S(B k ) will contain the mapped extremum points (cf ([3])), which are degenerate level contours. 
Therefore, the data induced partition will be updated also by the mapping of gradient lines. That 
is, the discovered extremum points are subtracted from X, which does not create new cells in 
14, but instead alters cells' topology. 




(a) The data induced partition before 
the update Vi ={V 1 1 ,V£} 




(b) and after the mapping of 
£2, yielding V2 = Vi 1 , and 
{Vf,VJ}=cc(V?\£2) 



Fig. 10. The data induced partition before and after updating to account for the mapping of an isoline £2. The updating consists 
of relabeling Vi to become V 2 X and replacing Vi with {V2 2 , V2 3 }. 

The problem of quantifying the information content of S(B k ) becomes equivalent to quan- 
tifying the relationship between the data induced partition and the topology induced partition. 
For this purpose, we will consider the conditional entropy metric ([4]) and define the conditional 
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entropy, H (A4\Vk), as the following function of the data induced partition and the topology 
induced partition. 

H (M \v t)= -j:j:^f>^^-^>. (,7) 

MieM V J eVk PV ; V\ v kJ 

An important property of how the conditional entropy evolves under a motion primitive based 
reconnaissance is stated in the next proposition. 

Proposition 4: Let Ai and Vk be respectively the topology induced partition and a data 
induced partition, and let Vk evolve according to ( fT6] ). Then the conditional entropy, H(Ai\Vk), 
is a non-increasing function of k. 

Proof: This result follows directly from Proposition [3} because 14 is a refinement of 14-1. 

■ 

By employing Proposition [T] and Proposition |4j the conditional entropy allows us to describe 
the evolution of the reconnaissance as iterative information acquisition. Initialization with no 
prior information is equivalent to V = {X}, and also H(M\V ) = H(M). (Note, that H{M) 
is unknown at the outset.) Under the reconnaissance, as more information is gathered, H(M\Vk) 
decreases, and assuming that H(M\Vk) approaches zero in the limit as k —> oc, the topology 
induced partition can be fully recovered from the collected data by merging cells of Vk as k 
increases according to fL5] ). 

C. Guiding reconnaissance through the conditional entropy 

Since the reconnaissance can be represented as an iterative information acquisition process, an 
efficient reconnaissance protocol should at each step utilize the acquired information to decide 
what motion program to apply next and where in the search domain to apply it. Therefore, our 
aim is to exploit the structure of the data induced partition as a means to guide the reconnaissance 
such that it collects relevant topological information. Collecting relevant information in terms 
of the conditional entropy will correspond to a refinement of Vk through isoline mapping which 
yields a strict decrease in the conditional entropy: 

H(M\V k )>H(M\V k+ i). 

As a first step to achieve this objective, the next theorem gives a connection between the 
structures of the data induced partition and the topology induced partition. 

Theorem 4: Let Vk be the data induced partition of the domain, X, under the search sequence 
B k applied to the scalar field / : X —> R. Define for every V£ G Vk the set 

M' (VI) = {M t eM (/, X) : M % n Vi ± 0} . (18) 

Then, it follows that 

\M'(v£)\>\-2x{Vi) + l\. (19) 

Proof: We start the proof by noting that S{B k ) contains the discovered extremum points, 
and therefore, according to ( [T5] ), there are no mapped extremum points in the cells of the data 
induced partition. Assume that V^ n £* = 0, V£* G Cr 0,2 (/, X), i.e. all extremum points are 
known and subtracted from X. Then, by applying the same argument as in the proof of Theorem 
[2j we can show that: 

\M'(V()\ = \-2x{Vi) + l\. 
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If, however, this assumption is not valid, i.e. there are extremum points that lie within the given 
cell (i.e. extrema that have not been discovered), the cardinality of |A^(V^)| will increase. ■ 
Having information about the number and locations of extremum points allows many con- 
clusions to be drawn regarding the topology induced partition. In what follows, we describe 
corollaries of Theorem [4] that establish properties of M under the assumption that all extremum 
points have been discovered. This assumption can be expressed in terms of Cr°' 2 (/, X) and 
S(B k ) as: 

Cv°> 2 (f,X)cS(B k ), (20) 

since S(B k ), in addition to the mapped level contours, contains the mapped extremum points. 
(See the definition in ([3]).) Moreover, taking into account ( [T5] ), this assumption will imply that 
all extremum points are reflected in the topology of the cells in the data induced partition. 
Specifically, given a mapped isoline contour £, and given that there are exactly K extremum 
points in its interior, the sum of the Euler characteristics of all cells of V k that also lie in this 
contour's interior is given by 1 — K. 

Corollary 3: For each V£ e V k consider the Euler characteristic x{V k )- Assume that all 
extremum points have been discovered (|20|), and define the set 



V k = {VieV k : X (Vi)<-\}. (21) 

Then the following holds: 

H( M\v k) = - E E "-^xy 1 1o& ^7M) • <22) 

k k 

Proof: If the reconnaissance has discovered all extremum points in X and they are included 
in S(B k ), the partition 14 will not contain cells with Euler characteristic 1. Moreover, by Theorem 
ffl if for a particular cell V/! G V fc , x(V^) = 0, then \M' (V;!) | = 1. This implies that there exist 
Mi e M(f,X), s.t. VI C Mi, which implies that 

MX) log2 ^) "°- 

Therefore, only elements of V k with Euler characteristic less than zero will have a contribution 
to the conditional entropy. ■ 

Corollary 4: Let Vk-i be a data induced partition with the set V' k _ 1 defined by ([21]). Assume 
that the isoline £ is mapped such that £ e V k _ x e V k _ v and it induces the update 

Vk = (v k - 1 \{vi_ 1 })u{vj: t v^}. 

Then assuming that both /x(V^), /x(VJ?) > 0, it follows that 

H(M\V k - 1 )>H(M\V k ). 
Proof: Note that 

+ 2^ 2^ -fix) log2 pM) • 

Mi&M ld{a,b} ^ y ' PV kJ 
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From the log-sum inequality this expression is equal to zero if and only if: 

and otherwise is strictly larger than zero. 

Note that i) according to Theorem MJ |A^ 7 (^-i)l > lj an ^ ii) every isoline lies within a 
single cell of the topology induced partition. Therefore, there is at least one cell Mi such that 

Mi fl V% = and M* n V£ ^ 0, or equivalently Mi n y fc a = and M* n V£ ^ 0. This contradicts 



fl23| ), which implies that: 

^(M|V,_!)>^(M|V,). 

■ 

Assuming that all extremum points are known, the information about the topology induced 
partition is localized in the cells of the data induced partition that have Euler characteristic 
xiYk) < 0- Therefore, a parsimonious reconnaissance procedure that does not acquire redundant 
information is to concentrate the isoline mapping in these cells, which, according to Corollary 
|4j yields strictly decreasing conditional entropy. In this way, given that all extremum points have 
been discovered, i.e. all areas that contribute to the conditional entropy have been identified, 
this procedure will continuously yield information about the topology induced partition of the 
function. However, if this is not the case, i.e. if there are undiscovered extremum points, isoline 
mapping carried out in accordance with the protocol of Fig. [6] will be insufficient to reconstruct 
the topology induced partition, and the agent will have to also apply gradient following to identify 
the remaining extremum points. 

This type of reasoning reveals a connection between reconnaissance and the paradigms of 
exploration vs. exploitation. In various fields of study, such as ecology [25 j, behavioral economics 
[26] and cognitive psychology ll27l . interrelated concepts of exploration and exploitation have 
proven to be useful of organizing discussions of human decision dynamics. The distinction 
between these phases of information acquisition have been well described by Cohen et al. ETll : 
"Decisions require an exploration of alternatives before committing to exploiting the benefits of 
a particular choice. Furthermore, many decisions require reevaluation, and further exploration 
of alternatives, in the face of changing needs or circumstances." All decision makers face the 
choice of "whether to exploit well-known but possibly suboptimal alternatives or to explore risky 
but potentially more profitable ones." 

In this sense, the analogy between the exploration vs. exploitation paradigm and the current 
set-up of unknown field reconnaissance can be revealed by considering the partition cells in V f k . 
The isoline mapping that subdivides these cells is equivalent to exploitation. On the other hand, 
the discovery of new critical points and thereby the addition of elements to V f k can be equated 
to the finding of new alternatives to be exploited. In this way, the mapping of gradient lines 
becomes equivalent to exploration. 

We further formalize this notion through defining a bound on the conditional entropy that can 
be evaluated from the perspective of the reconnaissance agent. 

Corollary 5: Assume that all extremum points (maxima and minima of the field) are known 
and define 

^ ( V *) = E ^7W lQ S 2 I" 2 * M) + A ■ ( 24 ) 

v£ev k 

Then, the following inequality holds 

H(M\V k )<H(V k ). (25) 
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Proof: Noting that the entropy is maximized when the partition is divided into equal parts, 
we can write 

\M f (v j )\ 

2-. ^nc\ log2 „ <vi\ - 2^ „/^iA^/T/ni log2 



MiGM 



KX) °- n(V{) ~ ^ H{X)\M'(V£)\ °'n(vi)\M'(vi 



On the other hand, with an argument similar to the proof of Theorem |4| it can be shown that: 

**) l0fe ,W)|A^)| - M30 lofe ^ W) + 1] ' 

_ ■ 

The metric if (14) purely depends on the data induced partition. Therefore, it represents a 
subjective quantification of the remaining uncertainty. The next theorem shows how it evolves 
under a reconnaissance protocol that restricts the isoline mapping motion programs only to the 
cells of V' k . 

Theorem 5: Let the search be conducted according to the search protocol shown in Fig. [6} 
and suppose that for every b k = b lS0 (r ), the point r is chosen within a cell V k _ ± e V' k _ l9 where 
V k _ x is defined by ([21]). Then, if b k = b iso , it follows that 

S F (V,)<F(V,_i), (26) 

and if b k = b 9rad , it follows that _ _ 

H(V k ) > tf(V*_i), 



where if (14) is defined by Q24 ) 



To prove Theorem |5j we state the following proposition. 

Proposition 5: Let £ be an isoline mapped according to the protocol described by the flow 
chart of Fig. 6l such that £ e V fc J L 1 e 14- 1, and which induces the update 

v k = {v*- 1 \{vi_ 1 })u{v?,v*}. 

Then, the topology of the cells V k and Vjf satisfies the following: 

x(V k a ),x(V k a )<0 (27) 

and 

Xm + x(V k b )= X (VLi)- (28) 

Proof: The search protocol shown in Fig. [6] dictates that all isolines are mapped starting 
from a previously mapped gradient line. This means that an isoline will always encircle at least 
a single previously discovered extremum point and as a result form two cells, both of which 
have Euler characteristics satisfying ( [27] ). 



On the other hand, ([28 
The proof of Theorem 



) follows directly from the Poincare-Hopf theorem. 
5\ can now be given. 
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Proof of Theorem \5j For ( [26] ) under isoline mapping, the rate of change in H(V k ) = H k 
yields 



+ £ 



i6{a,6} ^ V ' 



/f ™ log 2 (-2 X {Vl) + 1) , 



where it is assumed that V k) V k C Vj!_ v and it is taken into account that because of ( [27] ), it 
follows that 

-2x(^L 1 ) + l = |-2 X (^L 1 ) + l|>l. 
From the log-sum inequality, it follows that 



> 



V m * ; loe = V W fcj log 

Ji>^ g2 (- 2 ^)+i) i^w Nra (-2xra+D 

A/A V " (V * } 



iG{a,6} ^(X) 






> ^i og2 x 



H(X) (-2 X (VU) + 1) 



Therefore, 



E 



M ^ } log 2 (-2 X (^) + 1) < ^_T log 2 (-2x(^L 1 ) + 1) 



and respectively 

Hk < Hk-i. 

Under extremum search, finding a new extremum is equivalent to puncturing a hole, and 
respectively, decreasing the Euler^haracteristic of the cell containing this extremum by 1. When 
substituted back, this shows that H k is nondecreasing under extremum search, which concludes 
the proof. ■ 

This result establishes H k as a useful metric in the context of the exploration vs. exploitation 
paradigm. That is, H k is consumed under isoline mapping (exploitation) and increased under 
gradient line mapping (exploration). We will utilize this relationship to define reconnaissance 
protocols that are inspired by foraging. 

V. Machine reconnaissance and topology based feedback 

In this section, we present two reconnaissance protocols for identifying the topology induced 
partition of a particular unknown field. The first one employs the metrics developed above 
together with the topology of the cells in the data induced partition as a decision input. Hence, it 
utilizes them to decide where and with what motion program to search next. The other strategy 
under consideration will be called a scanning strategy, since it will not base next steps on 
anything inferred about the topology. These two strategies will be compared through Monte- 
Carlo simulations. From this perspective the scanning strategy can be thought of as the baseline 
solution to the problem against which the benefits of employing topology based feedback can 
be evaluated. 
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A. The topology feedback strategy 

We will base the topology feedback strategy on the consumption rate, 

Rk = H k _i — H k , 

which is utilized as a switching criteria. As established by Theorem [5} if isoline mapping is 
restricted to the cells of the data induced partition with Euler characteristic equal or smaller 
than —1, R k is strictly positive in the exploitation mode. In the exploration mode, it is either 
strictly smaller than zero (if the exploration mode yields the discovery of a new extremum), or 
equal to zero (if it does not). The heuristic that we propose is to exploit (map isolines), when 
the consumption rate lies within the set 

{R k >k~ T }U{R k <0}, (29) 

and to explore otherwise. The constant T > determines how aggressive the reconnaissance 
strategy is. The most aggressive behavior is equivalent to the rate being such that R k < k~ J \ 
Mk > 0. In this case, it is easy to verify that the search sequence will consist of alternating b lso 
and b 9rad motion programs. At the other extreme, a conservative reconnaissance will correspond 
to exclusive exploitation, i.e. for some time n, T can be chosen sufficiently large s.t. R k > k~ T , 
Mk > n. 

The rate R k provides a choice of a program to search with. The question that remains is 
at which point to initiate a particularly chosen motion routine. A consistent choice should 
guarantee that the conditional entropy, H(M\V k ), converges to zero. Under the assumption that 
all extremum points have been discovered, H k is a bound on the conditional entropy. Therefore, 
we will show that there exists a suitable choice of an infinite sequence of originating points, 
which guarantees that under the exclusive execution of b lso programs, H k goes to zero. 

Theorem 6: Assume that B k is generated according to the reconnaissance protocol described 
by Fig. [6J Moreover, assume that there exists n such that b k = b lS0 (r ), Mk > n, with the points 
r chosen at each instance according to the following sequence: 

• Define the segments l^ = V k n Q where V k e 14, Q e Q{B k ) (Q a gradient path), and let 
the constants A^, a^ associated with each segment be defined as: 

dij = inf /(r) 
Aij = sup /(r) 

rein 



Find a segment /*■ such that 

Choose the point, r such that r e l*j and 



/*• = arg max (A^ - a^) . (30) 

J I— Vint- 



f(A 3 ) + /( 



(lj n 



Then, the following holds: 
Proof: Define the metric 



/(r ) = i^lL_±^l 



lim H k = 0. 

k— 7>oo 



E E 

viev' k MjeM 



sup /(r) - inf /(r) 
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where V k is the set of cells in Vk having Euler characteristic < — 1 as defined by ( [21] ), and let 
N k be the cardinality of the set {V k D Mj : V£ eV k ,Mj e M}. Given that the reconnaissance 
is executed under the protocol described by Fig. [6j Prop. [5] dictates that every time a cell is split 
by isoline mapping, the sum of the Euler characteristics of the two new cells is equal to the 
Euler characteristic of the original, and moreover, neither of the two resultant cells have Euler 
characteristic of 1. 

Assume that one of the new cells formed by the isoline mapping has Euler characteristic of 
0, then this new cell will not be a part of V k+V and therefore, the metric L k evolves according 

to: 

(2N k - 1\ 

L « <L *HvH- (31) 



This follows from ( |30| ), i.e. the protocol dictates that the segment that is cut in half is the largest 
one. 

Another possible scenario is that both of the new cells have Euler characteristic smaller than 
0. Again, according to Prop. [5] this can only occur when the original cell has Euler characteristic 
smaller than —1 (the sum of the Euler characteristics of the resultant cells should equal to the 
Euler characteristic of the original one). In this case, L k does not change, but it is easy to see 
that: 

N k+1 = N k + l, (32) 

since V k+1 will have one more element than V k . Also from Proposition [5] it can be concluded 
that this scenario can occur only a limited number of times until all the cells in V k+1 have Euler 
characteristic of —1. That is, each cell V k e V' k9 such that x(^k) = n < —1, can be divided 
by a sequence of isoline mappings into at most n cells with ^ = — 1. Therefore, we can have 
Lk = L k+ i at most — x — 1 times. By defining C k as: 

for every case of / > fc, we can write: 

'OAF — 1 \ max [°^-^ _c 'fc] 

Li < Li 



n&>' k 



2N k 
Thus, as / —> oo, L\ converges to zero. Since /(r) is Morse function, 

v^eV k MjeM 

will also converge to as k —> oo and consequently so will H k . ■ 

To complete the specification of the topology guided strategy, we designate a random procedure 
for choosing the originating points for gradient tracking. That is, the initialization points for the 
b 9rad (r ) programs will be chosen at random with respect to a uniform probability density on the 
set of points on the mapped isolines S(B k ). The resulting reconnaissance protocol is summarized 
in Alg. [Tj Fig. [TT] shows the isolines mapped during a particular run of this topology driven 
reconnaissance strategy. It can be clearly observed that the mapped isolines are concentrated 
near the critical level sets associated with saddle points. 
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Algorithm 1 Topology guided reconnaissance strategy 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10: 

11 
12: 
13 
14: 
15 
16 
17 
18 



k = 

B k = {}. 

14 = {X} {Initialization with no prior information} 

loop 

if R k > k~ T or R k < then 

Choose y according to the sequence described in Thm. [6] 

Execute b k+ i = b lS0 {r ) to map £ 

V fc+1 = (V fc \K'})Ucc(^U) 
else 

Get y ~ U(S(B k )) (That is, choose the point at random from a uniform distribution 

on the set points on the mapped isolines.) 

Execute b k +i = b 9rad (r ) to map ( 

for All critical points r* uncovered by ( do 

v, +1 = (v fc \K'})uK'\r*} 

end for 
end if 

B k +\ = B k U {6/e+i} 
k = k + l 
end loop 




Fig. 11. The isolines mapped under a particular run of the reconnaissance strategy described by Alg.[T]for T = 0.5. 



B. Open loop scan strategy 

The scanning strategy that will be further utilized as a benchmark is a simple routine, which 
divides a mapped gradient line into n parts with equal range drop and maps the isolines that 
originate from them. It is formally described by Alg. [2} 



An illustration of the first several steps in executing this strategy is presented in Fig. 12 



After a gradient line is mapped, its range is partitioned uniformly into n parts (Fig. |12(a)| ). After 
mapping the isolines corresponding to this partition, the robot moves to a point randomly chosen 
on an unexplored part of the search domain and again maps a gradient line and then repeats the 
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Algorithm 2 Scanning reconnaissance strategy 



1 


k = 


2 
3 
4 
5 
6 
7 

8 


B k = {}. 
V k = {X} 
loop 

Choose r ~ £/(X) 

Execute b 9rad (r ) to map £ 

Find {ri <E ( : 1 < i < n} with /(r$) corresponding to a uniform partition of the range 

/(C) 

for z = 1 : n do 


9 
10 
11 


Execute 6 2SO (r^X) to map £ 

v k+l = (v k \{v>})ucc(vi\i) 

end for 


12 


end loop 




(a) (b) 

Fig. 12. An illustration of the initial several steps in mapping an unknown field through the n— scan strategy 



same isoline mapping routine (Fig. |12(b)[ ). 

C. Monte Carlo Validation 

To validate and compare these two reconnaissance protocols, we will use scalar fields that are 
realizations of a spatial gaussian process. Such fields are widely used in atmospheric modeling 
and data assimilation ll2Sll . and they have also been employed as underlying models for studies 
in unknown field reconnaissance (T). 

They are defined as follows: for each point r e X, the value of the function is considered as 
a realization of a random process /(r) ~ AT(0, 1), which for each two points r 1? r 2 G X has a 
correlation 



£[/( ri ),/(r 2 )] = exp 



|£i -r 2 | 
2d? 



(33) 



Reference [20] provides conditions under which the function /, as a realization of a random 
field, will be a Morse function with probability 1. In practical terms, we can calculate the 
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Fig. 13. The evolution of the conditional entropy for two different values of T: an aggressive strategy (T = 0.5) and a more 
conservative strategy (T = 2). 



realization of the field at discrete grid points, and then estimate the value of the field at each 
point of the domain through linear interpolation. 
In Fig. 



13 



we show the result of Monte-Carlo simulations of applying the topology guided 
strategy with two different values of T: an aggressive strategy corresponding to T = 0.5, and a 
more conservative strategy corresponding to a larger value, T = 2. Fig. 1 1 3(a) | plots the average 
of 

H(M)-H(M\V k ), 

for each instance k over 100 runs of the strategy, each on a different randomly generated field. 
Both strategies are initialized with V = {X}, which implies that H(M\V ) = H(M). An 
important observation is that the strategy with the large T is more efficient in the short term. 
This is due to the fact that the short term exploitation of the available resources gives an initial 
gain. However, the strategy with the large T is overwhelmed by the strategy with the small T 
in the long run, as the early investment in finding extremum points pays off and provides more 



area to be exploited as the reconnaissance progresses. Fig. |13(b)| gives the standard deviation of 
the conditional complexities of the two strategies. This shows that the more aggressive strategy 
in this case is also the more consistent in the long term. 

Fig. [14] presents the topology guided strategy compared with the benchmark n- scan strategy. 
The topology guided strategy can be clearly identified as the more efficient one. It consistently 
collects more information and achieves this with smaller dispersion. 



VI. Human guided reconnaissance 

In the previous section, we have proposed a protocol for unknown field reconnaissance, whose 
goal is to uncover the topological structure of the field. This design objective has been motivated 
by the need to efficiently identify a minimum set of features, whose mapping can yield a good 
qualitative and quantitative understanding of the underlying phenomenon. In this section, the 
focus shifts from the design of algorithms to the assessment of human decision making. In 
particular, the guiding conjecture will be that humans intuitively recognize the importance of 
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Fig. 14. A comparison of H(M) — H(M\Vk) comparing a topology guided strategy that uses feedback of information about 
the topological characteristics of cells currently in the data induced partition and an open-loop, n-scan strategy. 



the field's topology and therefore manifest a preference for discovering the topology induced 
partition. 

A. Reconnaissance game for testing human decision making 

To evaluate human-in-the-loop reconnaissance, we have designed a game, which simulates the 
mapping of a unknown field by a robotic vehicle that is under the supervisory control of a human 
subject. The subject is instructed that the hypothetical reconnaissance agent under her control 
can perform two types of tasks — the mapping of gradient lines and the mapping of isolines. 
Hence, the subject can utilize the same two search programs as the ones that are employed by 
the machine reconnaissance protocols discussed in the previous section. 



The scalar field is defined on a square search domain and it is generated according to ( [33] ), 
with two correlation lengths-d = 0.25 and d = 0.5 (given that the square has unit length sides). 
Fig. [15] illustrates that by varying the correlation length, we effectively control the complexity of 
the topology, and hence of the induced partition. Every subject is presented with multiple fields 
to map. The fields are chosen from a large archive with each field having a unique identifying 
number. Every even numbered field has high topological complexity (d = 0.25), and every odd 
numbered field has low topological complexity (d = 0.5). 

Every subject is instructed by a video [29] before the experiment, which assures the uniformity 
of the instructions. The basic points that are conveyed by the video are: 

• The game involves robotic reconnaissance of an unknown terrain. 

• The player teams up with a virtual robot that has been programmed to perform simple 
motions — mapping gradient lines and mapping isolines. 

• The responsibility of the player is to instruct the robot how to explore a specific area and, 
when she believes the acquired map of the given area is of sufficient quality, to command 
the robot to another one. 

• The subject is told that there is no wrong choice of what constitutes a sufficient mapping 
of a terrain. 



The game interface is shown in Fig. [16] The subjects are given 12 minutes for mapping. When 
the clock runs down to zero, the game automatically stops. Every time the subject clicks on the 
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(a) Low topology complexity (b) Hight topology complexity 
area (d = 0.50). " area (d = 0.25). 



Fig. 15. Two examples of high and low topology complexity fields. 



empty space, she automatically maps a gradient line (Fig. 20(b)). On the other hand, when a 



subject clicks on a gradient path, the result is the mapping of an isoline (Fig. |16(b)| ). Whenever 
a subject believes that the map is of sufficient quality, she clicks on the "Go to another area" 
button and starts mapping a new square domain (Fig. |16(c)| ). 
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(a) (b) 

Fig. 16. A basic depiction of the game interface as described in the text. 



(c) 



B. Topology feedback in human subjects 

As we have seen in Section [IV] acquiring topology relevant information is equivalent to 
continuously decreasing the conditional entropy. Therefore, if a subject is utilizing topology 
feedback with the goal of uncovering the structure induced by the critical points, she will have 
a preference for mapping isolines that reduce the conditional entropy. 

Corollary [3] established that the uncertainty about the structure of the topology induced partition 
is concentrated within cells of the data induced partition that-assuming all extremum points are 
removed from them-have Euler characteristic \ < — 1- This observation leads to a natural 
procedure for determining the existence of a preference for identifying the topology of the field. 
The data induced partition can be divided into two parts, V£, and its complement V& \ V%. In this 
division, the set V£ contains all cells of the data induced partition that have Euler characteristic 
less than or equal to —1, together with the cells that would have such characteristic under the 
discovery of new extremum points. In other words, V£ C V' k , where V' k is defined by ( [21] ). If 
all extremum points have been discovered, then V% = V k . It can be easily verified that every 
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time an isoline is mapped within V£ the conditional entropy is decreased, while if it is mapped 
within Vk \ V%, it stays the same. Therefore, the preference of a subject for mapping isolines 
within V£, as opposed to mapping isolines within its complement, will also imply a preference 
for identifying the cells of M. 

To quantify this propensity, we state the hypothesis that the probability of a subject clicking 
on a point r^ in a cell V belonging to V£ to map an isoline is given by: 



P p (ri eVe V° k ) 



(KK)V 



where /3 > is a positive constant for each subject. 

If )3 — 1, the probability of the subject mapping an isoline within a cell belonging to V£ 
is equal to its normalized area. In other words, the behavior of the subject is random, and 
she does not utilize any topology feedback. However, if /3 = 0, the subject exclusively maps 
isolines within V%, and in this way achieves continuous reduction of the conditional entropy. 
(The topology guided protocol described in Sec. |V-A| has (3 = 0.) Therefore, the preference of 
the subject towards identifying the topology can be directly quantified by the value of )3 9 with 
smaller values corresponding to higher preference. 

Assuming that the clicks are independent of each other, the probability of a particular sequence 
of isoline mapping clicks, 



where r^ corresponds to the position of the i th click, will be given by: 



f(ri,r 2 , 



•>-n i 



P^Ti) 



ifrieVeV O ki 

else 



(34) 



Here, we have used the indexing rii to denote the state of V£ immediately before the i th isoline 
is mapped. 

The parameter /3 can be estimated through a maximum likelihood estimator as 

P = argmaxP(ri,r 2 ,--- ,r m ). 



I Random Robot Players 
|Human Players 




0.2 0.4 0.6 0.8 



1 1.2 



Fig. 17. Histograms of the estimated parameter f3 for two populations — a human player population and a random computer 
player control population. 
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Fig. [T7] shows the histograms of /3 for two populations of players. The first population 
consists of 26 human players, who were subjects of the above described experiment. The 
control population consists of 26 "computer players", who conduct reconnaissance on the same 
fields that were mapped by the human players by mapping the same number of isolines. The 
difference between the control population and the human population is that the control population 
is programmed to map the isolines randomly by choosing the originating points from a uniform 
random distribution on the search domain. 



The statistics of the two samples^ whose histograms are shown in Fig. [T7J are as follows: the 
mean of the human population is f3 human = 0.32 and the standard deviation is a human = 0.15; 
the mean of the computer players population is f3 computer = 0.97 with standard deviation of 
a computer = 0.04. As expected, the computer population has a /3 characteristic close to 1, which 
corresponds to their random game strategy. At the same time, the small value of (3 for the human 
population is an evidence that the humans have a strong preference for discovering the topology. 

To confirm the difference between the random style of the computer players, and the style of 
the humans, we performed statistical significance testing. Kolmogorov-Smirnov statistics (K-S) 
was used to test the hypothesis that the two populations were sampled from the same random 
process. The K-S test rejects or confirms such a hypothesis based on the distance between the 
empirical cumulative distribution functions (CDF) of two samples. For the two populations under 
investigation, this distance was calculated to be 1. (The same can be also concluded from Fig. 
17 ). This distance corresponds to p — value = 6 x 10" 13 , which is the probability of achieving 
such extreme test statistic by chance, i.e. the probability that the two populations have been 
sampled from the same random distribution. To calculate the test statistic and its corresponding 
p — value, we used the kstest2 function of Matlab R2010a, which in turn has been developed 
according to ll30ll . 
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Fig. 18. The parameter f3 for each subject with respect to the topological complexity of the environment. 



From the statistical analysis of /?, it is clear that humans have a manifested preference for 
mapping the topology of the field, i.e. /3 < 1. This preference, however, is stronger in some 
subjects and weaker in other. Since an alternating sequence of low topology complexity maps 
and high topology complexity maps is used in the experiment, we can verify the consistency of 
the preference for each subject through computing /3 independently for the two types of fields. 
Fig. [18] shows the result of this analysis (the correlation between these two data sets is 0.78). 

We use this consistency to divide the subjects into three equal-sized groups based on their /3 
characteristic: bottom third (containing the third of the subjects with the lowest values of /5), 
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middle third and a top third. Each group can be viewed as having a different reconnaissance style, 
and we will analyze how the differences in these styles manifest through the defined information 
metrics. 
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Fig. 19. The evolution of (35) and its standard deviation as function of the number of clicks. 



Fig. 19 shows 



H(M)-H(M\V k ), 



(35) 



averaged over separate games played by the subjects in the bottom third and the top third group 
for each click k. In this case, we have used H(M) = H(M\Vo) (Vo = {X}) as a normalization 
factor to account for the fact that the different fields explored by the subjects have varying 
topological complexities. Also, to be able to account for the different lengths of the games, we 
have adopted standard game duration for each group. That is, the length over which the separate 
games are averaged in each group corresponds to the duration above which only 10% of the 
games continue. 

There are three observations that can be made: i) the subjects in the bottom third perform 
more efficient reconnaissance of the topology; ii) the subjects in the top third spend more clicks 
per area; iii) the standard deviation of the conditional entropy as a function of the number of 
clicks is higher for the subjects that use less topology based feedback. 

Fig. |20(a)| compares the same groups in terms of the entropy of the data induced partition. 
One can clearly observe that at the expense of spending more clicks per a field, the subjects in 
the top third achieve higher data induced partition entropy. 



Another interesting observation can be made through Fig. |20(b)[ The graph corresponds to the 
distance between the maximum possible partition entropy, log 2 |14|, and the actual one H(y k )\ 

log 2 |V*|-ff(V*). 

It also represents a metric of the uniformity of the size of the cells within the data induced 
partition. This metric initially grows for subjects in both of the groups. This corresponds to a 
concentrated mapping of isolines at the beginning (isolines associated with a single gradient 
line), which leads to few small- size cells and a large cell corresponding to the unexplored space. 
The initial growth gradually tapers off for the subjects in the bottom third. This can be 
explained by the fact that they pursue an understanding of the field's topology as opposed to 
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a detailed map of its shape. In other words, they parsimoniously try to acquire a good picture 
of the underlying structure induced by the critical points and, as a result, distribute the mapped 
isolines more uniformly around the search domain. On the other hand, the subjects in the top 
third acquire maps with a higher level of detail. They achieve this by methodically mapping 
higher numbers of isolines per each gradient path. 
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Fig. 20. The evolution of the entropy as a function of the number of clicks. 



VII. Conclusions and Open Problems 

The research described in this paper began as an attempt to understand decision dynamics in 
the execution of tasks carried out by mixed teams of humans and robotic agents. The specific 
focus has been the decisions that are required in carrying out reconnaissance missions-and more 
specifically in the exploration of unknown scalar fields. The paper has introduced formal models 
of information content in the kinds of spatially varying unknown fields. A Shannon-like entropy 
metric of information content of an unknown field was proposed, and it was shown how this 
metric could be related to the differential topology — and more specifically to the critical level 
sets of the field. In terms of this information metric, we were able to describe the way in which 
a reconnaissance mission acquired knowledge of the unknown field. By mapping gradient lines 
and isolines using the reactive control laws of our previous work (J5|, Q, [[SI), an increasingly 
fine partition of the reconnaissance domain was obtained. It is precisely the partition entropy of 
these partitions whose rate of increase can be used to define how effectively the reconnaissance 
is progressing. By defining a distinguished partition associated with the a priori unknown critical 
level sets of the field, we have been able to make use of certain conditional entropies to describe 
how much topological information is being acquired by the reconnaissance. 

In trying to understand the information needed to describe a unknown field defined on a 
compact domain, the critical level sets have turned out to be surprisingly important. While they 
clearly comprise a minimal set of essential qualitative features, the connection established in 
Theorem 3 with the entropy of the field came as something of a surprise. In view of this, 
the results of Section IV and the algorithm of Section V have been aimed at showing how a 
reconnaissance mission can be structured so as to climb an information gradient aimed at the 
goal of estimating all critical level sets. 

The final part of the paper reports results of an experiment in which human subjects (under- 
graduate engineering students) acted as mission directors in a simulated robotic reconnaissance of 
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an unknown scalar field. A measure of bias toward learning topological features of the unknown 
field is described, and in terms of this measure, we have observed that all our subjects showed 
some tendency toward trying to discover topological information — i.e. toward trying to learn the 
location of all critical level sets. Some subjects exhibited this tendency much more than others, 
however. Analysis of the game play showed that subjects who were strongly biased toward 
accumulating information about the topological characteristics of the field tended to be more 
parsimonious in terms of the numbers of isolines they mapped. They also tended to be more 
consistent from one game to another. The subjects who were biased toward seeking topological 
information exhibited a more even balance between numbers of isolines and number of gradient 
lines mapped. Subjects who were less biased toward topological information tended to acquire 
more general detail about the search domain by mapping large numbers of isolines. 

The results we have reported must be viewed as preliminary, but they point to a number of 
questions that seem worthy of further study. In terms of modeling information acquisition in 
problems of search and reconnaissance, there are many potentially important extensions that 
could be pursued. Perhaps the most obvious is to consider fields that vary in either time or 
space. All the work reported above has treated spatially stationary gaussian fields. In order for 
the models to capture the key features of real world reconnaissance problems, the correlation 



lengths of ( [33] ) should be allowed to vary over the domain. When this occurs, there are a variety of 
questions about which little is known. The trade off between metric and topological information 
will need to be studied carefully in such settings. In particular, if spatially non-stationary random 
fields are used in experiments with human beings, it will be of interest to learn what features 
of the field have greatest attraction for people directing simulated reconnaissance missions. Is 
there greater interest in finding and mapping the highest peaks or are regions of high variability 
in the field strength of greater interest? 

In any setting in which human decisions in the exploration of unknown fields are studied, it will 
be important to understand how acquired information provides cues regarding what to explore 
next. We have explored the way in which knowledge of the field's topological characteristics 
seems to inform next steps in the reconnaissance. Decision makers may be sensitive to other 
types of features as well, and research is needed to understand which characteristics of field 
variability will be most important in guiding a human mission director. 

A significant area that is open for future research is temporally varying fields — such as would 
be used to model a chemical cloud undergoing both diffusion and convection in the atmosphere. 
Nothing is currently known about the time variation of critical level sets or the partition entropies 
that we have studied. Moreover, in terms of decision making, time variation raises an entirely 
new set of questions for future study. 

Yet another question for the future is to understand how the reconnaissance of an known 
field should be carried out most effectively when multiple sensor-equipped robotic agents are 
available. The question is of special interest in the case in which the field is either spatially or 
temporally varying, but it is of interest in the stationary case as will. 

We conclude by saying that the research presented in this paper has raised more questions than 
it has answered. At this point our only definitive conclusion is that the process of information 
acquisition in the exploration of unknown fields is worthy of further study. 
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